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Abstract. Let K [x, y] be the algebra of two- variable polynomials over a field K . 
A polynomial p = p(x, y) is called a test polynomial for automorphisms if, whenever 
ip(j>) = p for a mapping ip of K[x,y], this tp must be an automorphism. Here we 
show that p G C[x, y] is a test polynomial if and only if p does not belong to any 
proper retract of C[x, y] . This has the following corollary that may have application 
to the Jacobian conjecture: if a mapping ip of C[x, y] with invertible Jacobian 
matrix is "invertible on one particular polynomial" , then it is an automorphism. 
More formally: if there is a non-constant polynomial p and an injective mapping 
ip of C[x, y] such that ip(ip(p)) = p, then tp is an automorphism. 



1 Introduction 

Let K[x, y] be the algebra of two- variable polynomials over a field K of characteristic 0. 
A subalgebra R of K[x,y] is called a retract if there is an idempotent homomorphism 
it of K[x,y] (called a retraction or a projection) such that ir(K[x,y}) = R. 

There are several equivalent descriptions of retracts of K[x,y] known by now: 

(i) K[x, y] = R © / for some ideal / of K[x, y]; 

(ii) K[x, y] is a projective extension of R in the category of ET-algebras; 

(iii) By a theorem of Costa every proper retract of K[x,y] (i.e., one different 
from K[x,y] and K) is of the form K\p] for some p = p(x,y) G K[x,y]. The authors 
earlier proved [5] that there exists an automorphism of K[x,y] which takes p(x,y) to 
x + y -q(x,y) for some q(x,y) S K[x,y], and every polynomial of the form x + y -q(x,y) 
generates a proper retract of K[x,y]. 

(iv) (see p(x,y) generates a retract of K[x,y] if and only if there is an endo- 
morphism of K[x, y] which takes p(x, y) to x. 

(v) (see (Sj) p(x,y) belongs to a proper retract of C[x, y] if and only if p(x,y) is 
fixed by some endomorphism of C[x,y] with nontrivial kernel. 

Recently retracts have found another application in a general setup of arbitrary 
free algebras and groups in relation with test elements, introduced in [Hj. In general, 
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an element g of a group or an algebra F is a test element if any endomorphism of F 
fixing g is actually an automorphism. It is easy to see that a test element does not 
belong to any proper retract of F\ a remarkable result of Turner ^Uj says that, if F is 
a free group, then the converse is also true. Thus, an element of a free group F is a 
test element if and only it does not belong to any proper retract of F. 

Here we establish a similar characterization of test polynomials in C[x,y]: 

Theorem 1. A polynomial p G C[x,y] is a test polynomial if and only if p does not 
belong to any proper retract of C[x, y\. 

Our proof uses several recent results, in particular, a result of Drensky and Yu 
[3] mentioned in the item (v) above. Crucial for our proof is the following result of 
independent interest. 

Theorem 2. Let p be an injective endomorphism of C[x,y] which is not an automor- 
phism. Suppose that <p{p) = p for some non-constant polynomial p € C[x, y]. Then 
p € C[q], where q is a coordinate polynomial of C[a;,y]. In particular, p belongs to a 
proper retract of C[x,y]. 

Recall that q = q(x,y) is a coordinate polynomial of C[x,y] if it can be taken to x 
by an automorphism of C[x, y\. 

We also use results of Shestakov and Umirbaev [Zj on estimating degrees of polyno- 
mials in two-generated subalgebras of K[x,y]. Another ingredient is a result of Kraft 
[H] concerning the subalgebra ip°°(C[x, y]) = n^ =1 ( / 9 00 (C[3;, y]). 

Theorems 1, 2 have the following corollary: 

Corollary. Let ip be an endomorphism of C[x,y] with invertible Jacobian matrix. If 
there is a non-constant polynomial p € C[x,y] and an injective mapping ip such that 
ip(f(p)) = p, then ip is an automorphism of C[x,y]. 

This strengthens our earlier result [SJ Corollary 1.7], where we showed that, if ip 
has invertible Jacobian matrix, then (p(p) = p implies that (p is an automorphism of 
C[x,y]. 

To conclude the Introduction, we raise a problem motivated by results of this paper: 

Problem. Suppose p € C[x,y] is a test polynomial and <p is an injective mapping of 
C[x,y]. Is ip(p) necessarily a test polynomial? 

It is interesting to note that, by a result of Jelonek [3], a "generic" polynomial of 
degree > 4 is a test polynomial. 

2 Proof of Theorem 2 

We consider the following two principal cases. 

Case I. There is a coordinate polynomial in (p(C[x,yj). 
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Case II. There are no coordinate polynomials in <p(C[x, y]). 
In Case I, consider two subcases: 

(1) (p is not birational, i.e., does not induce an automorphism of the field of fractions. 
Then, by a result of Kraft jSJ Lemma 1.3], (p°°(C[x, y]) is either C or C[/], where 

/ = f(x,y) is some polynomial. Obviously, if ip{p) = p, then p G cp°°(C[x, y]). We are 
therefore going to focus on the case ip°°(C[x, y]) = C[f] and show that, if <p is injective, 
then p G C[q], where q is a coordinate polynomial. 

Now suppose r = r(x,y) is a coordinate polynomial in ip(C[x,y}), and let r = 
ip(s{x,y)). Then, since 92 is injective, the polynomial s = s(x,y) must be coordinate, 
too, by the result of pQ. Therefore, upon changing generating set of C[x, y] if necessary, 
we may assume that r = (p(x). Furthermore, we can replace <p with its conjugate by 
an arbitrary automorphism, say a, i.e., with ifi = a<pa~ l , and at the same time replace 
p with p\ = a(p). Then we have: 

i>(pi) = aipa~ l {a(p)) = a(p) = p\. 

Therefore, the pair (ip,pi) has the same properties that the pair (<p,p) does, namely, 
ifi is injective but not birational, and ip(pi) = pi) in particular, p\ G ip°°(C[x, y]). By 
choosing a appropriately, we can also have ^(x) = x, thus getting x G t^°°(C[x, y]). 
Then, if p (and therefore pi) does not belong to C[q] for any coordinate polynomial 
q, ip°°{C[x,y\) cannot be of the form C[/], which is in contradiction with the result of 
Kraft mentioned above. This completes case (1). 

(2) ip is birational, i.e., induces an automorphism of the field of fractions. Again, as in 
the case (1) above, we deduce from that <p must take some coordinate polynomial to 
coordinate. Thus, upon changing generating set of C[x,y] if necessary, we may assume 
that ip takes x to u, and y to v- f(u), where C[u, v] = C[x, y], and f(u) is a non-constant 
polynomial (otherwise, cp would be an automorphism). 

Now let 

p = p(x,y) = ^2cijX l y J . 

Then 

<P(P) =^c ij u i v j {f{u)) j . 

i,j 

Let x r y s be the highest term of p(x, y) in the "pure lex" order with y > x. Then 
in (p(p), the highest term is that of (p{x r y s ) because the y-degree of <p{y) is not lower 
than that of </?(x). Furthermore, the highest term of (p(x r y s ) must have the y-degree 
at least s since otherwise, one would have both u and v of y-degree equal to 0, which 
is impossible. 

If the y-degree of (p(x r y s ) is > s, this gives a contradiction with (f(p) = p. Now 
suppose the y-degree of ip(x r y s ) is exactly s. This is only possible if the y-degree of v 
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is 1 and the y-degree of u is 0. Then, arguing as in the case (1) above, we may assume 
that (f(x) = x. Therefore, (p(y) = (y + g{x)) ■ f(x). Then from (p(p) = p we get: 

c ij x i y j = c ii x \y + 9{x)) j {f{x)) j . 

Again we use the "pure lex" order with y > x to focus on the monomial of highest 
degree on either side, but this time we compare the x-degrees of these highest-degree 
monomials. We see that these x-degrees cannot be equal unless f(x) is a constant, 
contradicting the assumption. This completes the proof in Case I. 

In Case II, we are going to prove the following somewhat stronger statement: 

Proposition. Let (p be an injective endomorphism of C[x,y], and suppose that there 
are no coordinate polynomials in (p(C[x,y]). Then ip°°(C[x, y\) = C. 

Proof. Let (p(x) = u = u(x,y), p(y) = v = v(x,y), and let D(u,v) denote the 
determinant of the Jacobian matrix of (p. Since (p is injective, D(u,v) ^ 0. Now there 
are two cases: 

(1) deg(D(u,v))= 0, i.e., D(u,v) is a non-zero constant. Then, by a result of Kraft [3], 
we have Lp°°(C[x,y]) = C. 

(2) deg(D(u,v))> 0. Note that for any k > 1, there are no coordinate polynomials 
in ip k (C[x,y]). Indeed, if there were a coordinate polynomial in (p k (C[x, y]), then, by 
the result of [I], there would have to be a coordinate polynomial in 99 fe_1 (C[x, y]). 
This would lead to a contradiction with the assumption that there are no coordinate 
polynomials in (p(C[x,y]). 

Let (p k (x) = u^ k \ v ?fc (y) = v^ k \ Then from deg(D(u, v))> and from the "chain 
rule" we get deg(D(u^ k \ v > k. Now the Proposition will follow from the lemma 
below. Before we get to it, we need one more definition. 

We call a pair (p, q) of polynomials from K[x, y] elementary reduced if the sum 
of their degrees cannot be reduced by a (non-degenerate) linear transformation or a 
transformation of one of the following two types: 

(i) (P> Q) — ► (p + A 4 ' Q k ! Q) f° r some fi G K*; k > 2; 

(ii) (p,q) — > (p,q + VP k )- 
Now we are ready for our 

Lemma. Let p = p(x, y) and q = q(x, y) be two algebraically independent polynomials 
such that the pair (p, q) is elementary reduced. Let n = deg(p) < m = deg(g); m, n > 2, 
deg(D(p, q)) > k. Let w = w(x,y) £ C[p, q\). Then, unless it; is a linear combination 
of p and q, one has deg(w) > min(n,k). 

Proof. The proof here is based on a result of Shestakov and Umirbaev Theorem 
3]. Let N = N(p,q) = gc 2( n , m ) ~ m - n + deg{D(p,q)) + 2. Following 0, we may 
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assume that the highest homogeneous parts of p and q are algebraically dependent; 
otherwise, deg(w) > n is immediate (unless w is a linear combination of p and q). 
Then g c ci(n m) ~ m ~ n — ^' in deed, if g.c.d.(n,m) = n, then the pair (p, would 
not be elementary reduced, contradicting the assumption. If g.c.d.(n,m) < n, then 
— -ft r > 2, therefore — > 2m, hence ™" . - m - n > 0. 

g.c.d.(n,m) — ' g.c.d.(n,m) — ' g.c.d.(n,m) — 

Thus, from now on we assume N = N(p, q) > deg(D(p, q)) + 2. 
Suppose now that the y-degree of w = w(x, y) is of the form - - m ^ • b + r ^ 0, 
where < r < — r. Then, by [71 Theorem 31, we have 

— g.c.d.(n,m) ' •> »** J' 

deg(u>(p, q)) > b ■ N + mr. 

If 6 ^ 0, this implies deg(itf(p, (/)) > > fc + 2 > fc. If b = 0, then r / 0, implying 
deg(u;(p, g)) > m > n. 

It remains to consider the case where the y-degree of w = w(x, y) is 0. Then the 
x-degree of w must be nonzero; suppose it is of the form - - m - ) • b\ + r\ ^ 0, where 
— ri < g .c.d!(n,m) ■ Tnen > again by Theorem 3], we have 

deg(u>(p, <?)) > i>i • N + nr\. 

As before, b\ ^ implies deg(ui(p, g)) > /c. If 6i = 0, then r > 2 because we assume 
that w(x,y) is not linear. Then we have deg(w(p,q)) > 2n > n, which completes the 
proof of the lemma. □ 

Continuing with the proof of the Proposition, we aim at showing that for any integer 
M, there is an integer k such that the degree of any polynomial in (p k (C[x,y]) is > M. 
The above lemma "almost" does it if we use it withp = ip k (x) = u^ k \ q = f k (y) = v^ k \ 
but it has one extra condition on the pair (p, q) to be elementary reduced, whereas a pair 
may not be elementary reduced. However, if we denote by (uW,i)W) an ele- 
mentary reduced pair obtained from (u^ k \ v^) by elementary transformations, we shall 
have all conditions of the lemma satisfied for this pair while obviously C\u^ k \v^] = 
C[u^ k \v^]. In particular, the inequality deg(D(u^ k \v^)) > k follows from the fact 
that the mapping x — > u^ k \y — > is a composition of (p k with an automorphism 
a of C[x,y] in such a way that a is applied first. Therefore, the "chain rule" applied 
to this composition yields deg(D(u^ k \v^))= deg(D(u^ k \ v^)). Thus, our lemma is 
applicable to the pair (u^,!)^)), which completes the proof of the Proposition and 
therefore of Theorem 2. □ 

3 Proof of Theorem 1 and Corollary 

The "only if part of Theorem 1 follows from a result of jH] rather easily. If p = p(x,y) 
belongs to a proper retract C[q] of C[x,y], then, by 9 , for some automorphism a, 
a(p) belongs to C[x + y ■ u] for some polynomial u = u(x,y). Then the mapping 
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x — > x + y ■ u, y — > fixes the polynomial x + y ■ u, and therefore also fixes a(p). Thus, 
a{p) is not a test polynomial, and neither is p. 

For the "if part of Theorem 1, suppose that p does not belong to any proper retract 
of C[x, y], and let ip(p) = p for some mapping ip of C[x,y]. Then, by the result of [3], 
<p must be injective. Then, by our Theorem 2, ip must be an automorphism, hence p is 
a test polynomial. □ 

Proof of Corollary 1. By way of contradiction, assume that (f is not an automor- 
phism. Then, by our Theorem 2, p € C[q], where q is a coordinate polynomial of 
C[x,y]. Therefore, the composite mapping ipip fixes a polynomial f(q) in q. Then it is 
easy to see (by looking at the highest degree monomial in f(q)) that ip{(f(q)) = c ■ q 
for some c € C*, which implies, by the result of pQ, that <p(q) is coordinate. A map- 
ping of C[:c,y] with invertible Jacobian matrix that takes a coordinate polynomial to a 
coordinate polynomial is obviously an automorphism, a contradiction. □ 

In conclusion, we recall a result of |;SJ Theorem 1.3] saying that if, for a mapping ip of 
C[x,y] with invertible Jacobian matrix, (p(x) generates a proper retract of C[x,y], then 
ip is an automorphism of C[x, y]. Then, the case where (p(x) belongs to a proper retract 
but does not generate it, can be ruled out since in that case, ip(x) = f{p{x,y)), where 
p(x,y) generates a proper retract of C[x,y], and / is some one- variable polynomial of 
degree >1. The gradient of such a polynomial cannot form a row of any invertible 
Jacobian matrix, which can be easily seen from the "chain rule" applied to f(p(x,y)). 

Therefore, by Theorem 1 of the present paper, if ip is a counterexample to the Ja- 
cobian conjecture for C[x,y], then ip(x) must be a test polynomial. Perhaps a way to 
prove the Jacobian conjecture for C[x, y] could be through showing that the gradient of 
a test polynomial cannot form a row of any invertible Jacobian matrix. This is known 
to be the case with (non-commutative) partial derivatives of a test element of a free 
group of rank 2, see Corollary 2.2.8]. 
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